Solving Nonlinear Estimation Problems Using Splines

W e describe the use of splines for solv-i n g n o n l i n e a r model estimation problems, in which nonlinear functions with unknown shapes and values are involved, by converting the nonlinear estimation problems into linear ones at a higher-dimensional space. This contrasts with the typical use of the splines [1]–[3] for function interpolation where the functional values at some input points are given and the values corresponding to other input points are sought for via interpolation. The technique described in this column applies to arbitrary non-linear estimation problems where one or more one-dimensional nonlinear functions are involved and can be extended to cases where higher-dimensional non-linear functions are used. The benefit of using the approach described here is obvious. Many real-world systems can only be appropriately modeled with nonlinear functions, while the estimation problem is much simpler if only linear functions are involved. It is thus highly desirable if a nonlinear estimation problem can be transformed into a linear estimation problem at a different space. In this column we use the cubic spline (i.e., piecewise third-order polyno-mials) [1], [2] to illustrate the technique. However, the same approach can be used with other types of spline as illustrated at the end. We demonstrate the applications of the technique in signal processing and pattern recognition with an example. RELEVANCE The topics presented here extend the standard cubic spline interpolation algorithms by finding a direct relationship between the interpolated values and those at the spline knots (i.e., the given or to be estimated input/output points) for an arbitrary nonlinear function. This direct relationship can be formulated as an inner product of location-dependent weights and the values at the knots. The approach presented here may find practical applications within pattern recognition, classification, system combination , speech recognition, and signal processing. Some existing applications are briefly discussed. PREREQUISITES The prerequisites consist of basic calculus and basic linear algebra. Optimization techniques could also be useful but not necessary. BACKGROUND Splines are piecewise or multiple-segment functions with pieces or segments connected smoothly with each other, where the connecting points are called knots or control points. Splines are typically used to approximate the values of a nonlinear function y 5 f1x 2 within the range 3x 1 , x N 4 by interpolating the val-u e s a t k n o t s 5 1x i , y i 2| …